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ABSTRACT es 
Two. studies were donesabout the selection of specific 

neasures for use*in institutional goal translation and evaluation 
processes; in addition,| they prcevided prototype methodologies for 
setting objectives. The} first, "Institutional Goal Achievement: 
Measures of Frogress"s (IGA), suggested by its results that the 
process of setting goals and weasuring progress would benefit from 
consensus-building techniques, whether in individual institutions, 
apong institutions of one type, or across institutional types. The 
second, The Higher Education Outcome Measures Identification Study 
‘(OMIS), studied and developed procedures for collecting outcomes and 
associated seasures deemed most important to different types and 
levels of decisiontakers. The procedures of. these two studies can be 
gga in goal translation, and, in addition, important insights ‘can 

be gained about the outcomes inforration preferences and needs of 
\postsecondary. education participants and constituents when the 
measures are used as research tccls. (Author/MSE) 
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Introyuction 


-In a recent essay on.university goals, Conrad (1974) -ppints out.that goals: 
(1) are standards against which to judge success, (2) provide a source of 
legitimacy which justify the activities of an organization, (3) define organi- 
zational needs and priorities, (4) define production units for “outputs” for the / 
organization, (5) define the organization's clientele, and (6) define the a ae 
of die colaeidaants between the organization and societys He also points out 
that "in most universities, goals are often implicit, residing in an extended” 
body of collective understandings eather than in explicit statements." He 
goes on to say, "If university goals are to sérve the purposes listed above, 
they must be identified acre precisely.” : rs : . | 

The goals of specific colleges and ‘universyties have traditionally been 
described in broad, vague, “high sounding," And aftan internal ty ficortstetent 
terms. Such a situation did not cause apy major Problems as long as people fully 
accepted the intrinsic value of a colfege education and as long as‘ the demands 
made on postsecondary education institutions remained retatively stable. * How- 
ever, during the fifties and sjxties, postSecondary educational institutions, 
in particular those ,in the g6llegiate sector, Came under increasingly severe 
pressure to grow in sizeyvand complexity and to meet the diverse needs, desires, 
and expectations of ma y new groups throughout society. In response to these 
voluminous ,- diverse/demands being made on them, institutions often found Cheneel ves 

‘everything to evérybody." In the process, colleges-and 

an j 


universities of all types grew increasingly alike; they tended to lose their 


individual fdentities as they pursued an Mdeal- typified by the university model. 


g - ' % “¢@ 
As ‘a “result, the purposes, goals, objectives, and prio ties of institutions 
‘ naa hot! been very distinct, no very apparent even to’ ‘students, faculty, and - 


“administrators. my f 


e 


os a a 


It \'s apparent that the problem of bait ckeeh ted Medarves dive ef identifying ‘ 
and clearly stating: (1) what it is the institution is trying to accomplish; 
(2) whom it is trying oral (3) how it intends to /aecomplish its objectives;: ‘ 
cand (4) how much and how Well it intends to accomplish. Without answers to ” 


these questions an institution cannot really specify its purposes and goals 


sufficiently to meet its internal needs for dimgctions plus. it becomes difficult 


i to stake gut its claim to some.’special place, in the larger postsecondary 
on : ; ’ < ° 
“education community. And unless institutions get down to tangible, concrete 


We i 


Apvels of ‘specificity, it is impossible to really answer these questions. The 
fey to are the problem of goaJ-setting is "concreteness" or "specificity." 
“There seems to be widespread recognition of the problem, but limited capability 
w yt 0 Heal with it. Institutions have found that dealing in the abstract with the 


‘ “e 
“e development of goals and objectives has proved * be a very frustrating experi- 
a 


pec: they can often identify and state their nats in genera) terms but cannot 


«puke the crucial leap to translating their goals into objectives in terms 
te 

oo 

3 pectic, enough. to be effectively acted upon and assessed. Developmental work 


re 


', 
nt 


aoe nducted by NCHEMS at institutions such as South Dakota State University and 
- 
katamazoo Valley Communi ty Gariege a hi the development of institutional 


and aver ane tana unit goals has further emphasized the need for, concrete 
guidelines and alternative strategies: ‘for deciding about goals agd’ especially 
‘for, translating thee general goals into specific objectives. 

‘a Translating goals into measurable objectives to be pursued by the institution 
is the step that gives concrete and specific expression to the institution's 
purposes. It is also a ait that is espectattY difficult to achieve. 
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In translating goals into measurable objectives, all of the issues and 


complexities related to outcomes identification and measurement come into play. 
Inability to state institutional purposes in specific, measurable asl ad pttee 
hinders, if not BYCET IES assessing the extent to which goals he been ‘ 
achieved. Moreover, Tack of specificity. also increases the difficulty of ef- 
‘ ; fective communication with institutional constituencies, especially external 
u centtitcenries from whom pressures for snettevttonal accountability arise, = 
In eranstatina institutional goals into measurable objectives, , major 
j difffculty is identifying the pieces of evidence that demnigtrate progress toward 
the achievement of the established goals. One fa}tor contriputing to this 
— difficulty is that the persons responsible for identifying outcome measures 
most.aften “start from scratch." Furthermore, as in the goal-definition process, 
there is often difficulty in reaching agreunerit on specific‘ measures because 
of the variety of interests, perspectives, and expectations. 

The purpose of this paper is to describe two studies which .researched® . 
guest fons related to selecting specific measures to be used in gbal translation 
and evaluation processes. In addition, these efforts provided prototype 
methodologies that could be seid by institutional personnel to support the 
objective setting process. These studies were: . 

@ Institutional Goal Achievement: Measures of Progress (Romney, 1976) 


| 


A? r 
@ ‘Higher Education Outcome Measures Identification Study (Micek and 


ifrer. 1974). 


, These ‘studies/and their implications for.the goal translation process | 


~ are discussed in the following pages. 


Institutional Goal Achievement. (I iho: Measures of Progress 
; . The IGA study was quided by the following purposes ; 


e Assess the interests of three campus constituencies (faculty, admin- 
istrators, and trustees) in.bétne involved in institutional goal 
setting and assessment processes} \ : 

@ Examine the acceptability of specific goal ahéas (or planned-outcomes) 


to these three audiences in six kinds of institutions; 


Assess perceptions as to which goal areas can be measured and how they 
“can be measured,*as viewed by trustees, administrators, and faculty; 
e Gather information regarding perceptions of appropriateness of specific 
J ; kinds. of information as measures of progress toward the achievement 
of broadly-stated institutional. goals; 
. ® Determine if, the appropriateness of the measures of progress ‘toward 
the achievement of institution] goal areas varies significantly 
‘among, the three constituent groups; = 
e Determine if the appropriateness of the measures of progress for different 
institutional goal area’ varies significantly among six types of : 
institutions. : bad 

The study population consisted of 1150 faculty, trustees and administrators 
from 45 colleges and universities throughout the country. These institutions 
were Classified into six major categories, as follows: 

@ Public Doctoral-Granting Institutions. (7) 

@ Private Doctoral -Granting Institutions ‘(3) 

@ Public Comprehensive Universities and Colleges (9) 

t e Private Comprehensive Universities and Colleges (4) 
e Liberal Arts Colleges (12) 
@ Two-Year Colleges and Institutions (10) 
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Faculty, dnd trustee respondents to the study were chosen randomly whereas | 
administrators were selected based on their Specific institutional responsibility, 
The survey instrumént fe whigh these individuals were asked pasha consisted 
of i principal. sections. The firgt contained demographic and identifying 
information about the responderit. The second dealt ‘with measures of progress , 


.for twenty broadly-stated institutional goal areas. The goal areas were derived 


from the Institutional Goals Inventory (IGI) developed by and used with permission’ 


peru Educational Testing Service, (ETS). TheSIGI consists of several speciféc 


/ goal statements in each of twenty institutional goal aredy, thirteen of whitch are 


3 \edesignated as outcome goal areas; The remaining seven are labeled process 


+ 


\yoal areas by ETS. . Wir " 


i 
Associdted with each IGI goal area ‘on the study questionnaire were Sets 


of four to eleven measures of progress ie jtems of information to be 


used to demonstrate progress toward the jevement of the goal area in question. 


~ 


. All in all, respondents, in addition to the twenty goal-areas, were asked to 


‘respond to 125 measures ‘of progress, These measures were developed and 


~contributed’ by the staffs of NCHEMS, WICHE, and the Higher Education Center 


of the School of Education at the University of Colorado. In addition, a panel 
of experts, identified by the author and- staff members at NCHEMS; .reviewed and 
supplemented the list of measures, 7 

To complete the questionnaire, respondents were asked to rate the degree 
to which each goal area should be a goal of his or her institution. Secondly, 
the respondent was aekad to indicate the degree to which each item of 
information was considered to be an appropriate measure of progress for’ the 
“goal aiea. y ‘ - 
The'study produced a wealth of information, only a smalf portion of which 


has been thoroughly examined. A detailed examination of the results and 


conclusions obtained thus far are recorded elsewtiere (Romney, 1976), Only those, 
results. that rehate directly to the topic at -hand, sianely sevansiaeton of 
institutional goals into measurable objectives, are reported here. 
yj Generally, the appropriateness of measures of progress for.demonstrating 
institutional goal achievement did not. vary across the faculty, trustees, 
and administrators. There were only- seven instances in ahich views of 
the three types of respondents differed significantly, Trustees and 
administrators differed only with regard to two ness’ 

Q) Scholarly works produced by students and/or former seitinte that 

are considered suitable for publication (trustees lower than 
>” administrators), 

A2) ‘Institutional policies and procedures developed to protect the 
exercise of academic freedom by faculty, studente (trustees lower 
than administrators). : 

Trustees and faculty ratings of appropriateness differed only for one 
measure, Institutional policies and procedures developed to protect. 
the exeretse of acadenic freedom by faculty aiid studente (faculty 
significantly higher than trustees). Fe 
Admini travers and faculty ratings of the appropriateness of measures 
of progress differed significantly only with regard to five measures: 
Sattsfactton of currently enrolled students = recent graduates 


with their academic development (administrators ‘rated this as 


being more appropriate than did faculty), 


Student/Yaculty rattos (faculty rated this measure higher than did 


administrators), 
Amount of release time granted to. facelty members for community 


service (faculty higher than administrators), 


| 8 
a” 
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(4) Avaitabtltty Sar ad of an institutional information system 
(administrato# ratings higher than those of faculty members), 

UJ 7 ; * ‘ 
(5) Satisfaction of atudenta and former students: with the extent 


’ * set 
and nature of their educational experience and subsequent 


employment (administrators higher than faculty), 


The results suggest that if the following fourteen types’ of information 
were collected, progress could be demofistrated toward the achievement 
of the seven most important goal areas for each of the six institutional 
types. Moreover, this evidence would be acceptable generally to at p 
least the faculty, administrators, and trustees in each type of 


s 


institution. 


( ') Student ability to apply knowledge 


( 2) Continuing active tntellectual involvement of former ahciienin othér 
than formal, advanced study 
3) Course offerings and institutional opportunities pe ‘taining to 
the development of individual wodid. values, and pe*gonal growth 


Students and/or former ‘students expressing concern far hwnan 


welfare and well being e 
Employer satisfaction with former student vocational t4r professtuonal 


\ 


training ae \ 


¥ 


Seholarly works produced by graduate students and/or farmer 
} i. 
graduate students constdered suitable for publication | 


oe 
7) Basie research publications or other results of séholarly effort 
. produced by students or faculty members hivten the past’ year’ 
8) Evaluations and perceptions of sativie oF the community regarding 
the quality of institutional services avatlable to them 
9) Extetence of special courses wall Braiioies to meet the needs of 


particular groups of stuu:nts 
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Institutional policies and procedures developed to protect the 
exercise 0 * academik freedom by fpoulty and Stale 
Attendance and. partieipation by facujty in the, faculty senate 
or. stmilar body 
Faculty and staff percepttons and evaluations of internal morale 
Student and/or faculty attendance at cultural. activities 
sponsored by the inatitution 

(14) . Impacts of modifications made in courses did programs 


@ An examination of the fourteen measures listed above reveals a decided 
hack of tompatibility with measure§ currently used: to provide evidence 

© OF productivity. For example, in a study conducted to determine which 
“outcome measures" of: community colleges are collected by state agencies, 
Kinnison (1975) found that most are required to supply information on 


4 


they number of students enrolled, full-time equivalent students, and 
Wag j : 


degrees granted. To such a list one could add "grade-point averages” 
and "student/facul ty ratios" as typically ‘collected measures. Yet none 


of these measures was rated as Kighly appropriate in this study... I 


‘fact, the measures now in use tend to conform more in orientation and 


\ 
-subStance to those réjected by the participants. This apparent 


conflict can have some important implications for institutional manage- 
ment and produgtivity, as well as for statewide“control and coordination. 
If the theme of enumeration survives as the way to provide‘ evidenc® 

of accountability, institutional administrators and faculty may begin 

to. operate in accordance with radically different incentive structures. 
Emphasis on quality and impact would pale in the light of “degree 
production and "body counts." Indeed, it is suspected that most . 
incentives would operate to maximize degree production. and body counts, 
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In such a situation, all participants tend to lose in terms of impact 
‘ t 
and satisfaction. The measures found to be appropriate pieces of evidence 


Filia an empirically substantiated base for such a change. 


e In a different vein, the results of the study suggest sonie conclusions 

: regarding the development of goals and measures of progress for these 
goals. Respondents from six tyfes of institutions from all parts of 
the country were participants ™ this study. In general, it can be 
said that consensus was reached across all types of institutions as” 

‘to the appropriateness of some goal areas and ees of progréss 
within these goal areas. In some cases, consensus regarding goals and 
their measures was restricted to agreement within particular institutional 
types. Therefore, it is suggested that the process of selecting 
institutional aga isand appropriate measures of progress can utilize 
consensus-bui ding -techni ques and that these techniques are useful 
within individual institutions, within systems of similar types of 
institutions, and across a conglomeration of several types of 
-institutions. The number of goals and appropriate measurés thought 
to be appropriate most likely will decrease as the diversity of 
institutions involved increases. Yet the task of identi fying perceptions 


“of common purposes and of how to measure progress toward their 


achievement seems to be feasible as well as necessary. 


“© Higher Education Outcome Measures Identification Study (OMIS) 
, fay 


The. OMIS was initiated primarity. to study and develop, procedures for 


collecting outcomes and associated measures deemed most important to different 
types and levels of deeision makers. . The objectives of the study were to: 
Learn what outcome information different decision makers need for 


their decision-making responsibilities; 


il 
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e Learn what outcome information currently is available to “them. 
e Identify a high priority list of outcome measures. for which data -* 


ww e 
acquisition procedures need to be developed. 


OMiS“<urvey questionnaires were sent to 388 randomly selected college’ 
administrators (including 97 presidents, 97 chief academic aptiners. 97 
Abhele student. affairs officers and 97 chiéf fiscal officers) and 125 state- 
a level decision makers (including 75 statewide planners and 50 state legislators). 
_ The overall response rate for the study was 58%. Responses werereceived 
from at least one administrator in 95 of the 97 ingtitutions included in the 
pool. Whereas 235 (or 61%) of the college administrators returned completed 
“OMIS questionnaires, only 64 (or 51%) of the state-level participants responded. 
-The participants in the study were asked three types of questions. The 
first question ‘concerned the extent to which they felt they needed information 
about each of the ten outcome information categories delineated in Section I 
of the questionnaire (see Micek and Arney, 1974). The second major question 
dealt withthe extent to which each respondent felt the need for each of the 
specific outcome measures Visted in Section II of ‘the questionnaire. The fjnal 
question pertained to his/her, access to or ability to obtain. each of dhs 
outcome measures presented in Section II. Detailed results of the OMIS 
project are reported elsewhere (Micek and Arney, 1974): Following are a few 
excerpts from those findings that ave Uivectly related to the goal translation 
process. » ; , . ‘ 
‘@ Three measures fn the OMIS effort were given a-high "Need To Know" 
«é endorsement by all$ix groups. They were: 
(1) Number of students graduating from the institution after a certain 


4 


period of ,time as a percentage of their entering class. 
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(2) Mouber and percentage of graduates for the veer sin: eranetenred 
from another ccigel:. wet rn . 

(3) Number and percentage of students leaving the institution prior 
to receiving a degree, diploma, or certificate during a particutar 


academic term or year. F : je 


In addition, five measures were given a high - “Need:To Know" endorsement 


U 
‘ 


by at least five of the six groups: - , 

(1) Number and percentage of students surveyed who are taking noncredit, 
independent study, or special courses. 

(2) Number of students receiving a degree, diploma, or certificate 
within a certain time period. | 

(3) Average amount of time it takes a student to earn a degree, 
diploma, or certificate. * 

(4) Student scores on a scale measuring their degree of satisfaction 
with their progress in achieving their occupational career goals. 

(5) Number and percentage of former students (graduates and nongraduates) 
surveyed who were employed within a certain time period after . 
leaving the institution. | : 

interesting similarities and differences occurred among the six , 

respondent groups. With the exception of the chief fiscal administrators 

group, the top ranked measure for each of the groups was in the outcome 

area--entitled "Student Educational Career Development." The following 

identifies some highlights with respect to each of the groups: 


(1) Presidents--This group endorsed the widest range of outcome 


measures. Possibly this is a result of the wide range of decision- - 


$ 
making responsibitities encountered by people in this group. Two 


measures were unique to this group. 
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(2) Academic'Affairs Administrators--This group appears to have ‘the 
| rns aaa ee 
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t greatest need for measures relating to Student Knowledge, and Skills 
Development, Student Educational Career Development, and the 
Development of New Knowledge and Art. To some extent, this was 
to be laceptud, given the planning and management responsibil tties 
of respondents in this group. . 

(3) Student Affairs Administrators--Respondents in. this group "appear. 
to have concentrated on*measures concerning Student suceetonat 
Career Development, Student Educational Satisfaction, and Student 
iceuinatbonal Career Development, respectively. Of all the groups, 
this group indicated the greatest need for Educational Satisfaction 
{ measures. It is interesting to note that of the five measures 
unique to this group, four jstines stun waiencthens about their 

, educational progress. ‘ This may suggest that Student’ Affairs 
Administrators are more noveptive to this kind of information. 

(4 Budget and Finance hininistrators+-As expected, respondents in 

‘this group favored. those measures directly linked to financial. 

data. Further, each of the five measures that were endorsed ohty 

by this group concerned the stiereiie of Henngra data. 

a State-Level Planners--This cl ad of respondents endorsed many of ; 
oe same measures ‘that the respondents jn the four institutiona? | 

1 decision-maker groups endorsed ee Student Educational 
Career Development. Of all the groups, this group favored the 
greatest number .of measures associated with Student Occupational 
partes beactoumenk: , , mn 

(6) State Legislators--This arose concentrated its major attention 


on “measures associated with Student Knowledge and Skills Depetaneee 
14 ‘ 
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Student Educational Career Development, and Student Occupational 


* Career Development. 


Implications for Goal Translation 


As stated at the outset, goal translation is the process of stating 


institutional goals in Spectres _Measurable terms; that is, stating them in 


terms of measurab]e@ chdootives! Thi¢ goal translation process can be dis- 


aggregated into the following, greatly simplified steps: 


1) 


2). 
3) 


4) 


4) 


“@ 


‘“ 


Identification Of the full range of goals for thé organtzation. , 
Selection of those goats that will be most important. 

Identification of ‘the kinds of information that could be used to assess 
the degree: to which the high priority goals are being accomplished, .» 
Selection of the information items (e.g., outcome and environment 
measures) that will be unee to assess levels of goal attainment. 
Preparation of objectives Shated in terms of fhe information items 
selected previously as ‘well as the audience .for whom they are intended 


and the minimum performance levels expected. 


The reader should recognize that the goal translation process, a very 


complex exercise indeed, is only partially, supported by the study procedures 
ay 


and findings-recounted previously. For example: 


e Both the IGA and OMIS procedures can be useful to assist decision 


makers in the goal selection (Step 2) atid measure selection (Step 4) 
procedures outlined previously. 

The questionnaires and results of both studies may serve as resources 
‘during the goal and measure identification processes (Steps | and 3 
respectively). Clearly, however, there are numerous sources of goals 
and measures that might be referenced. Relying on these two studies 


would limit the scope of the goal translation exercise. 
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‘ @ Individuals involved in the goal translation process skould consider. 


measures of the types preferred by decision makers and postsecondary 


education participants in the IGA and OMIS studies, i.e., measures 
2 ° * 


ats 


jof measurable: impact and satisfaction. 
\ 


In conclusion, the. authors suspect that not only are the procedures of 


these two studies useful for addressing portions of the goal translation problem, 


*% but also when used in individual organizational units, institutions, systems 
or other populations as research tools, important insights can be gained’ about 


the outcomes information preferences and needs of postsecondary education 


r participants and constituents. . ‘ 
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